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Abstract Systems prone to faults are often equipped with a controller whose aim 
consists in restricting the behaviour of the system in order to perform a diagnosis. 
Such a task is called active diagnosis. However to avoid that the controller degrades 
the system in view of diagnosis, a second objective in terms of quality of service is 
usually assigned to the controller. In the framework of stochastic systems, a possible 
specification, called safe active diagnosis requires that the probability of correctness 
of the infinite (random) run is non null. We introduce and study here two alterna- 
tive specifications that are in many contexts more realistic. The notion of (y, v)-fault 
freeness associates with each run a value depending on the discounted length of its 
correct prefix where the discounting factor is y. The controller has to ensure that the 
average of this value is above the threshold v. The notion of a-resiliency requires that 
asymptotically, at every time step, a proportion greater than a of correct runs remain 
correct. From a semantic point of view, we determine the equivalences and (non) 
implications between the three notions of degradations both for finite and infinite 
systems. From an algorithmic point of view, we establish the border between decid- 
ability and undecidability of the diagnosability problems. Furthermore in the positive 
case, we exhibit their precise complexity and propose a synthesis of the controller 
which may require an infinite memory. 


Keywords Stochastic systems - Partial observation - Fault tolerance - Diagnosis 





The work of serge Haddad was supported by the project ERC EQuallS (FP7-308087). 





N. Bertrand 
Univ Rennes, Inria, CNRS, IRISA 
E-mail: nathalie.bertrand @inria.fr 


S. Haddad 
LSV, ENS Paris-Saclay, CNRS, Inria, Université Paris-Saclay 
E-mail: haddad@lsv.fr 


E. Lefaucheux 

Univ Rennes, Inria, CNRS, IRISA 

LSV, ENS Paris-Saclay, CNRS, Inria, Université Paris-Saclay 
E-mail: engel.lefaucheux @inria.fr 


2 N. Bertrand, S. Haddad, E. Lefaucheux 





1 Introduction 


Diagnosis. The designer of a system aims at eliminating faults that could trigger 
unwanted behaviours. However, for embedded systems interacting with an unpre- 
dictable environment, the absence of faults is not a reasonable hypothesis. Thus di- 
agnosis, whose goal consists to detect faults from the observations of the run of the 
system, is a crucial task. One of the approach frequently used to analyse diagnosabil- 
ity (i.e. the existence of a diagnoser) consists in modelling the system by a transition 
system whose states (depending on the internal part of the system) are unobservable 
and events may, depending on their nature, be observable or not. A diagnoser must 
fulfill two requirements: correctness and reactivity. A diagnoser is correct if it never 
erroneously claims a fault. It is reactive if every fault is announced after a finite delay. 
For finite systems, the diagnosability problem is decidable in polynomial time! while 
the synthesis of a diagnoser may require an exponential time [8]. 


Active diagnosis. Embedded systems are often equipped with one (or more) con- 
troller(s) in order to maintain some functionalities of the system in case of a patho- 
logical behaviour of the environment. It is thus tempting to add to the controller a 
diagnosis task. Formally some of the observable events are controllable and consid- 
ering its current observation, the controller chooses which subset of actions should 
be allowed to make the system diagnosable. A system is said actively diagnosable if 
there exists a controller ensuring the role of diagnoser. In [11], the authors showed 
that the active diagnosability problem is decidable in doubly exponential time. Then 
in [7], the authors designed a single exponential time algorithm and proved the opti- 
mality of this complexity. 


Probabilistic diagnosis. In transition systems, the unpredictable behaviours of the 
environment are modelled by a nondeterministic choice between the possible events 
from the current state. However, in order to quantify the risks induced by the faults 
of the systems, the designer often substitutes to the non deterministic choice by a 
random choice or equivalently by a weighted one. Then the model becomes a dis- 
crete time Markov chain in the passive case (i.e. without controller) and a weighted 
transition system in the active case (i.e. with a controller). The reactivity require- 
ment is then adapted by requiring that almost surely (i.e. with probability 1) a fault 
is announced [12]. The passive probabilistic diagnosability is a PSPACE-complete 
problem [4] while the active probabilistic diagnosability is an EXPTIME-complete 
problem [2]. 


Active diagnosis and degradation. However the choices performed by the controller 
ensuring active diagnosis may have a pernicious effect: to detect faults, the controller 
sometimes could favour the occurrence of these faults! Aiming to manage the degra- 
dation of a system, a controller ensuring safe active diagnosis ensures the diagnosis 
task and a positive probability that an infinite run is correct. A quantitative version of 





1 In this paper, we assume some familiarity with basic complexity notions, and refer the interested 
reader to [9]. 
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this requirement fixes a probabilistic threshold € to achieve. Safe active probabilistic 
diagnosability is undecidable; however, when limited to finite memory controllers, 
the problem becomes decidable in NEXPTIME [2]. 


Contributions. Ensuring a positive probability for correct runs is only one possible 
way to express a requirement on the degradation control of a system and it is not 
necessarily appropriate for all contexts. For instance, some systems are designed to 
correctly behave for a long period of time at the end of which they will be replaced 
by a new system. In order to address such requirements, we introduce two new spec- 
ifications of degradation control: 


— A system is (y, v)-fault free if, when applying a temporal discount y < 1, the 
mean value of the discounted length of the maximal correct prefix of a run is 
greater or equal to v. The qualitative version of this specification, called lasting 
fault freeness is obtained for y = 1 and v = co. This means that the average 
length of the maximal correct prefix of a run is infinite. 

— A system is a-resilient for a < 1 if the proportion of correct runs decreases 
asymptotically slower than a factor a at every time step. There are two qualitative 
versions of this specification: a system is strongly resilient (resp. weakly resilient) 
if for all a < 1 (resp. there exists œ < 1 such that) it is a-resilient. 


First we study these specifications in a passive framework. More precisely we focus 
on the qualitative notions. We establish that the safeness of a system implies its lasting 
fault freeness and its strong resiliency and that no other implication exists between 
the three notions. However they coincide for finite systems. 

Then we analyse the active framework. We show that diagnosability combined 
with (y, v)-fault freeness or with a-resiliency is undecidable. Afterward we improve 
the complexity result related to safe active diagnosis with finite memory showing that 
the problem is EXPTIME-complete. Contrary to safe active diagnosis, diagnosability 
combined with (1) lasting fault freeness, (2) strong resiliency or (3) weak resiliency, 
reamins decidable and more precisely is EXPTIME-complete. Moreover, we estab- 
lish that the additional constraints of lasting fault freeness and strong resiliency co- 
incide in the active framework. Those decidability results are all the more surprising 
since the corresponding diagnosers may require infinite memory. 


Organisation. In Section 2, we define the probabilistic transition systems and intro- 
duce diagnosis and the different specifications of the degradation of these systems. 
We also present the links between the qualitative versions. In Section 3, we establish 
the decidability status of the active diagnosability problems and, when decidable, 
their complexity. We then conclude and give perspectives of this work in Section 4. 


2 Diagnosis and degradation of a probabilistic system 
2.1 Probabilistic labelled transition system 


We introduce a standard probabilistic model of discrete time event system based on 
discrete time Markov chains (see [1]). 
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Definition 1 (pLTS) A probabilistic labelled transition system (pLTS) is a tuple A = 
(Q, qo, X, T, P) where: 


— Q is a countable set of states with gy € Q being the initial state; 
— X is a finite set of events; 

-TC Qx Xx Q isa set of transitions; 

P is a function from T to Q>o verifying: 


Vq€ Q, 5 Plg,a,q7]=1. 


(a,g')eZxQ|(qg,a,q')eT 


A pLTS is a labelled transition system (LTS) enhanced by probabilities on the 
transitions. The transition relation of the induced LTS is defined by: q + q for 
(q,a,q') € T; such a transition is called enabled in state q. By definition, in every 
state q of the pLTS, at least one transition is enabled, i.e. a pLTS is live. 


Notations. Given a countable set E, we denote Dist(E) the set of probability distri- 
butions on E. Let q € Q, the function associating P[q, a, q’] with a pair (a, q’) if 
(q, a, q’) € T and 0 otherwise, is an element of Dist(X x Q). The support of a distri- 
bution p € Dist(E), written Supp(p), is defined by Supp(p) = {e € E | p(e) > 0}. 
Thus The support of the above distribution is {(a,q’) | (q,a,q') € T}. When the 
support of a distribution is a singleton {e}, we denote this Dirac distribution 1e. 


Fig. 1 An example of (finite) pLTS. 


Example I A pLTS is represented by a labelled oriented graph whose vertices are the 
states and edges are the transitions labelled by the associated event and its probability. 
In Figure 1, the edge from qo to fı is triggered by the event f with probability 2, We 
will often omit the probabilities when they are equal to 1, and, more generally, when 
the distribution on the transitions exiting a state is uniform. 


We now introduce some important notions and notations used in the sequel. A 
run p of a pLTS A is a (finite or infinite) sequence p = qoaoqı ... such that for all 
i > 0, qi € Q, a; € X and when q;+1 is defined, q; 25 qi+ı- The notion of a run 
may be generalised by allowing to start in an arbitrary state q. We write N2 for the 
set of infinite runs starting in qo, assuming the pLTS A is clear from context. A finite 
run p ends in a state denoted last(p) and its length, denoted |p|, is the number of 
events in p. Let p = qoaoqı - . . qn be a finite run and p’ = gndndn41-.. a (finite or 
infinite) run starting in the last state of p, we call concatenation of p and p' the run 
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PP! = qoGoq1 --- Qn@nGn41---- The run pis called a prefix of p', which we will write 
p < p’, if there exists another run p” such that p’ = pp”. The cylinder generated by 
a finite run p is the set of infinite runs extending p: Cyl(p) = {P € 2 | p < p'}. 
The sequence associated with p = qaoqı ... is the word wy = aga, ..., and we write 


q 4 or q =e (resp. q 4 q or q =e q’) for an infinite (resp. finite) run p. A state 
q is reachable (from the initial state qo) if there exists a run p such that qo 4 q, also 


written go = q. The language of a pLTS A is the set of infinite words labelling runs 
of A and is formally defined by L“ (A) = {w € ©” | J qo 2, Le 





Forgetting the labels and merging (and adding up the probabilities) the transi- 
tions with same source and destination, a pLTS becomes a discrete time Markov 
chain (DTMC). In a DTMC, the set of infinite runs of À is the support of a proba- 
bility measure extended from the probabilities of the cylinders by the Caratheodory’s 
extension theorem: 


P 4 (Cyl(qoaogi . .n)) = Pla, ao, qı] tee Pldn-1, an—1; qn] - 


When A is fixed, we will often omit the subscript, and write P for P 4. Let p be a finite 
run, with a small abuse of notation we write P(p) for P(Cyl(p)). If R is a countable 
set of finite runs such that no run is prefix of another, we write P(R) for ` cp P(p) 
which is consistent as the intersections of the associated cylinders are empty. 


2.2 Partial observation and ambiguity 


In order to formalise the problems related to fault diagnosis, we partition the set of 
events X in two subsets X, and »%’,,, the observable events and unobservable ones, 
respectively. Moreover, we distinguish a special event, the fault f € Xu. 


Example 2 The set of events of the pLTS of Figure 1 is defined by Xo = {a,b} and 
Sy. = {f, u}. Transitions labelled by unobservable events are represented by dashed 
edges. 


Let w be a finite word on the alphabet X, its length is denoted by |w| and 1 
represents the empty string. The projection of words of Z* on the observable events 
is inductively defined by: 7(1) = 1, for a € Xo, m(wa) = m(w)a and for a € Xu, 
m™(wa) = r(w). We write |w|, for the observable length of w, i.e. [n(w)|. When 
w is an infinite word on X, its projection is the limit of the projections of its finite 
prefixes, and by convention |w| = oo. A pLTS A is called convergent with respect to 
a partition X = X, W Xy, if, from every reachable state, there is no infinite sequence 
of unobservable events: L” (A) N ©* 5” = Ø. When A is convergent, for all w € 
LY(A), r(w) € XY. In the sequel, we assume that the pLTS are convergent. We use 
the terminology sequence for a word w € X* U &”, and observed sequence for a 
word w € X3 U XY. The projection of a sequence is therefore an observed sequence. 

The observable length of a run p denoted |p|, € N U {oo}, is the number of 
occurences of observable events: |p|, = |wWplo. A signalling run is a finite run 
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Go@o41 ` ‘‘ An—14n such that an—ı is an observable event. The signalling runs are 
precisely the relevant runs from the point of view of partial observation as every ob- 
servable event gives an additional information on the run to an external observer. 
In the following, SR denote the set of signalling run and SR, the set of signalling 
runs of observable length n. Since the pLTS are convergent, for all n > 0, SR» is 
equipped with a probability distribution defined by assigning the measure P(p) to ev- 
ery p € SR». By convention the empty run qo is defined as the single run of length 0. 
Let w € &* be an observed sequence, we define its cylinder Cyl(w) = w and the 
associated probability P(Cyl(w)) = Pp € SR | T(P) = w}), often abbreviated 
by P(w). 

We now classify the runs depending on the occurrence of faults. A run p is faulty 
if the associated sequence w, contains f, otherwise it is correct. Let n € N, we write 
F, (resp. C,,) for the set of infinite runs such that the signalling prefix of observable 
length n is faulty (resp. correct). We define the sets of finite (resp. infinite) faulty and 
correct signalling runs F (resp. Foo ) and C (resp. Coo). Without loss of generality, by 
considering two copies of every state of the pLTS, we suppose that the state space Q 
of A is partitioned between correct and faulty states: Q = Q fWQe such that the faulty 
(resp. correct) states are only reachable by faulty (resp. correct) runs. An infinite 
(resp. finite) observed sequence w € XY (resp. 27) is ambiguous if there exists an 
infinite correct run (a correct signalling run) p and an infinite faulty run (a faulty 
signalling run) p’ such that x(p) = z(p’) = w. Otherwise it is either surely faulty or 
surely correct depending whether n~t (w) N SR C F or w~!(w) A SR C C. Arun 
is ambiguous, surely correct or surely faulty if its observed sequence is ambiguous, 
surely correct or surely faulty respectively. 


Example 3 Consider the pLTS of Figure 1. The correct states are gg and qı while the 
faulty states are fı and f2. The run pf = qof ( fia)” is faulty and ambiguous as the 
single correct run pe = qou(qia)” has the same observed sequence a“. For every n, 
the finite sequence a” is ambiguous while the sequence a”b is surely faulty as b does 
not occur in pe. 


2.3 Diagnosability 


Diagnosability of a pLTS is defined in terms of probability of sets of runs. Toward 
this goal, we define FAmb,, the set of infinite faulty ambiguous runs. 


Definition 2 (Diagnosability) A pLTS A is diagnosable if P(FAmb,.) = 0. 


Let n be an integer, FAmb,, is is the set of infinite runs of À which signalling prefix 
of observable length n is faulty and ambiguous. We recall the following result which 
allows us to use an alternative definition of diagnosability. 


Lemma 1 ([4]) Let A be a pLTS. Then lim P(FAmb\FAmb,,) = 0. Moreover, 
if A is finitely branching, then lim, P(FAmb,, \ FAmbas) = 0 and consequently 
À is diagnosable iff lim sup,,_,,, P(FAmb,,) = 0. 
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The alternative definition of diagnosability given by Lemma 1 is used implicitly 
multiple times throughout this paper. Among other things, it allows to synthesise a 
diagnoser (with infinite memory) in a simple way when the (finite) pLTS is diagnos- 
able. After an observed sequence w, the diagnoser claims a fault if w is surely faulty. 
By construction, the diagnoser is correct and since lim sup,,_,,, P(FAmb,,) = 0, it is 
reactive. In fact, we can build a diagnoser using finite memory by only remembering 
the current possible states and claim a fault when all these states are faulty [12]. 


Example 4 Consider the pLTS of Figure 1. FAmb,. is a singleton reduced to the run 
pf = dof(fia)” with a null probability. Thus this pLTS is diagnosable. FAmb,, = 
Cyl(qof( fia)" fi) U Cyl(qof ( fia)” f2) = Cyl(qof( fia)” * fi). The probability of 
FAmb,, is thus equal to a and converges to 0 as announced by the previous 
lemma. In this particular case, the diagnoser does not require any memory and claims 


a fault at the first occurrence of event b. 





2.4 Degradation 


We describe and study here three notions of degradation of a system: safeness, fault 
freeness and resiliency. A pLTS is safe if it guarantees a positive probability of infinite 
correct runs. We can refine this notion by quantifying it: a pLTS is e-safe if this 
probability is greater or equal than €. 


Definition 3 (Safe pLTS) Let À be a pLTS. 


- Fore > 0, Ais e-safe if P(C) > €; 
- Ais safe if P(C% ) > 0. 


As pointed out in the introduction, in some cases, safeness is a too strong require- 
ment. We formalise now two alternatives: fault freeness and resiliency. Fault freeness 
aims at quantifying the period of time during which the pLTS is correct. In order to 
(possibly) take into account the importance of the immediate future, we introduce a 
discount factor y < 1 for counting this duration. The expectation of this discounted 
value is then compared to a threshold v. 


Definition 4 (Fault free pLTS) Let A be a pLTS. 


- For0 < y < 1 and v € (0, oo], Ais (y, v)-fault free if 57,5, P(Cn)y” > v. 
- Ais lasting fault free if it is (1, 00)-fault free. 4 


Observe that when y equals 1, 5,5, P(Cn)y” is the mean observable length of 
the maximal correct signalling prefix of a random run. This justifies the name lasting 
fault free for an infinite expectation. 

The notion of resiliency is an alternative measure of degradation based on a factor 
degradation ratio per time unit a < 1. A pLTS is a-resilient if the proportion of finite 
correct runs which stays correct on the next occurrence of an observable event is 
asymptotically greater than a. This requirement has two qualitative variants: strong 
resiliency (resp. weak resiliency) requires a-resiliency for every (resp. for at least 
one) a < 1. 
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Definition 5 (Resilient pLTS) Let A be a pLTS. 
- For 0 < a < 1, Ais a-resilient if lim suPp o BEJ = 0; 
— Ais strongly resilient if for all 0 < a < 1, Ais a-resilient; 
— Ais weakly resilient if there exists 0 < a < 1 such that A is a-resilient. 


E ai a, p2 a, p3 








l 

f,1— pı f,1— po} f,1—p3 
I 
J 


l 
| 
| 
| | 
t LA er 


Fig. 2 An example of infinite pLTS. 


Example 5 The pLTS A of Figure 2 has a single correct run p = qoaqiaqz2 ... while 
every faulty run contains an infinite number of b. À is thus diagnosable. Moreover, 
the probability of p is equal to [[,,., pn and the probability of its prefix of length 
nis equal to r, = [],<,, pi. Consequently, A is safe iff limno rn > 0. By direct 
application of the definition, À is lasting fault free iff 57,5, Tn = co. Let us consider 
different values of (p; Jien- = 


ele. 


e Let pi = +. Then rn = Thus A is not safe but is lasting fault free. For every 


i+1 n+1° 
a <1, lim, ..(n + 1)a” = 0. Thus A is also strongly resilient. 
e Let p; = rir Then rn = wa: Thus A is neither safe nor lasting fault free. 


For every a < 1, lim, _(n + 1)?a” = 0. Thus A is strongly resilient. 
e We inductively define two sequences m4 and nx by: 
Np = 225<k 5 (hence no = 1) and Mk = ny + Dh Mj + nj. 

Define: 

-= Ik = [nk +X jer Mi +25, Der Mj + nyl; 

- Jr = bare Mj + Nj, Nr À 2 ep Mj + nl. 
When i € Ip, pi = $. When i € Jp, pi = 1. 
Observe that for all n € Jp, rn = 27 Zi<k Mi, 
Consequently >), "n > X k>o es Tn = D x>0 Dir Mig Lien = o, 
Thus A is lasting fault free. E 
Let n = 3 5<, mj + nj. Consequently, rn = 27 Xise Mi, Fix a = + 


J 
a = 22 jer Mi (V2) ase m+n; > 2m (4/2)—2mx =1. 


Tr. 
Therefore A is not a-resilient. 


The next theorem establishes the relationships between the qualitative versions of 
the three degradation notions. Note that the pLTS from Example 5 serves as witness 
for the last two statements. 
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Theorem 1 Let A be a pLTS. 


(a) If À is safe then A is lasting fault free and strongly resilient; 

(b) Assuming A is finite, A is safe iff A is lasting fault free iff A is strongly resilient; 
(c) There exists a lasting fault free pLTS that is not strongly resilient; 

(d) There exists a strongly resilient pLTS that is not lasting fault free. 


Proof We first prove item (a). Assume A is a safe pLTS. There exists € > 0 such that 
for all n, P(C,,) > €. Thus, >. P(Cn) > 32,51 € = œ. On the other hand, for 


alla < 1, limno Pie j < lim, 2 = 0. We conclude that À is lasting fault 





free and strongly resilient. 


To prove item (b), we pick A a finite pLTS. Observe that every bottom strongly 
connected component (BSCC) of A (here seen as a graph) either contains only correct 
states or contains only faulty states. Accordingly, we can refer to them as faulty BSCC 
or correct BSCC. As A is a finite Markov chain (with events labelling the transitions), 
almost surely an infinite run reaches a BSCC and the mean time to reach a BSCC is 
finite. Due to the first result, A is safe iff there exists a correct BSCC that is reachable 
from the initial state. 

Suppose that A is not safe. 


— Every reachable BSCC is faulty, and this implies that the mean time to reach a 
faulty BSCC is finite. This mean time is an upper bound on the mean observable 
length of the maximal signalling prefix of a run. Thus A is not lasting fault free. 

- We write m = |Q|. For all q € Qe, there exists pq a run starting in q composed 
of an elementary run from q to a faulty BSCC followed by an elementary run (or 
circuit) in the BSCC of which only the last event is observable (by convergence). 
This run has an observable length smaller or equal to m. We note uq, the proba- 
bility of that run and u = mingeg, Hq. Consider a signalling run p of observable 
length n for an arbitrary n and ending in q € Qe. From the existence of pq, 
P({p! € SRngmMC | p < p'}) < (1—p)P(p). Thus P(Cnm) < (1—H)P(Cn). 
So, P(C) € O((1 — u)™ ). Choosing a = (1 — y)", A is not a-resilient and 
thus not strongly resilient. 











This concludes the proof. 





3 Control and diagnosis 
3.1 Active diagnosis 


The extension of the pLTS formalism allowing to express control requires to fix at 
least two features of this formalism: the nature of the control and the distribution 
of probabilities of the controlled system. Controllable Labelled Transition System 
(CLTS) were introduced in [2]. In order to specify the control, a subset of observable 
events is considered as controllable. The control strategy forbids a subset of con- 
trollable events depending on the sequence of observations it has received so far. In 
particular it cannot change its control in between two observations. The transitions 
of the system are no longer labelled by (rational) probabilities but rather by (integer) 
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weights which represent their relative probabilities. Given a state and a set of forbid- 
den controllable actions, in order to obtain a probability distribution on the allowed 
transitions, the weights of the outgoing transitions labelled by uncontrollable or al- 
lowed controllable actions are normalised. Provided that the control strategy does not 
create any deadlock, the so-obtained controlled obtained is a pLTS. 


Definition 6 (CLTS) A Controllable Labelled Transition System (CLTS) is a tuple 
C = (Q, qo, X, T} where: 


— Q is a set of states with an initial state go € Q; 

- X = Xo W Xu is a finite state of events partitioned into the set of observable 
events Xo containing controllable events Xe C Xe and the set of unobservable 
events X, containing the fault f; 

-T:Qx 2x Q — Nis the transition function that associates an integer weight 
with each transition. 


For CLTS, the transition function T simultaneously plays the role of the probabil- 
ity function and the transition function in pLTS. We use weights instead of probabil- 
ities in cLTS, since due to the control normalizing the weights is anyway necessary. 

A CLTS has an induced transition system which transition relation is defined by 
q + q¢ if T(q,a,q') > 0. the extended relation = is defined as for pLTS. As for 





pLTS, we assume that the CLTS is convergent and live (i.e. Yq dq < q^. 
Example 6 A CLTS C is represented in Figure 3. The weights of the transitions, all 


equal to 1, are omitted. The only controllable event of C is b. The observable yet 
uncontrollable transitions (here on event a) are bold. 


Fig. 3 An example of CLTS. 


We now formalise the ingredients necessary to define the control of the CLTS. Let 
X° C Sand q € Q, let us write G~” (q) for the sum of the weights of the transitions 
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exiting q and labelled by an event of X°. Using this sum, we define a normalisation 
of the transition relation restricted to the events of X° by: 





reg) [EE ire € Band Ta) >o 
a 0 otherwise 


A strategy of a CLTS C is a function ø : Z* — Dist(2*) such that for all w € Z* 
and all X° € Supp(o(w)), X\ Xe € X°. Given an observation, a strategy consists in 
(randomly) choosing a subset of allowed events containing the uncontrollable events. 
Let C be a CLTS and o be a strategy, we consider the configurations of the form 
(w,q, X°) € XX x Q x 2” with w the observed sequence, q the current state and X° 
the set of allowed events by o after observation of w. We inductively define the set 
Reach, (C) of the reachable configurations under o by: 


- for all X° € Supp(o(1)), we have (1, qo, X°) € Reach, (C); 

— for all (w,q, X°) € Reach, (C) and all a € Xu N X° such that q & q’, we 
have (w,q', X°) € Reach, (C), and the corresponding transition is denoted by 
(w, 9, 2°) So (w,g, I°); 

— for all (w,q, X°) € Reach, (C), all a € X, N X° such that q & q’ and all 
X°’ € Supp(o(wa)), we have (wa, q', X°’) € Reach, (C), and the correspond- 
ing transition is denoted by (w, q, X°) “+, (wa, q’, 2°). 


A strategy o is called live if for every configuration (w,q, X°) € Reach,(C), we 
have G~* (q) 4 0. Only the live strategies are relevant as the other strategies create 
deadlocks. We are now in position to introduce the semantic of a CLTS controlled 
by a live strategy o in terms of a pLTS. Its set of states is Reach, (C) augmented by 
an initial state whose goal is to randomly choose in accordance with (1) the initial 
control. The probability distributions are based on T~ * if the current control is X° 
combined with the random choice of ø in case of an observable event occurrence. 


Definition 7 Let C be a CLTS and o be a live strategy, the pLTS C, induced by the 
strategy o on C is defined by Co = (Qo, X, Goo, To, Po) where: 


- Qc = {doc} U Reach, (C); 
— for all (1, go, X°) € Reach,(C), (doo, u, (1, go, X° )) € To; 
= for all (w, q, X°), (w’,q’, X’) € Reach, (C), 
((w,g, Z°),a, (w’,q’, 2°) € To iff (w,q, 2°) So (w',q’, 2°); 
— for all (1, qo, X°) € Reach, (C), Po (qoo, u, (1, Go, 2°)) = o(1)(2° ); 
for all ((w,q, Z°),a,(w,q,Z°)) € To and all a € XN X’, 
Pz ((w,q, 5°), a, (w,g,2°)) = T?" (q,a, q'); 
for all ((w, q, X°), a, (wa, q', z) € T, andall a E€ Xo N X”, 


Po ((w.9, 5), a, (wa, 5*)) = TE" (ad): o(w-a)(2*). 


Example 7 Consider the CLTS C depicted in Figure 3. There are two possible al- 
lowed subsets X and X \ {b} that we denote X7. Let us define the strategy o by 
ola”) = pn: X +rn X with ph + rn = 1 for all n € N and o(w) = 1» for 
every other w. A part of the pLTS C, is represented in Figure 4. Let us develop the 
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distribution of probabilities exiting the configuration (1, q1, X). The two transitions 
exiting qı are enabled with equal relative probabilities, thus normalised to 0.5. Since 
a and b are observable, the new control is chosen, in the case where a a is observed, 
by a probabilistic choice pı - X7 + rı - X while if a b is observed, there is a deter- 
ministic choice 15. This result in three transitions with probability 0.5p,, 0.571 and 
0.5 respectively. 
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Fig. 4 An example of controlled CLTS. 


Let us define the problems of active diagnosis in the context of the degradation 
control. Roughly speaking, given a CLTS, one asks whether there exists a strategy 
such that the associated pLTS is diagnosable and satisfies the required property re- 
lated to degradation. We distinguish, as usually done, the quantitative problems (i.e. 
including numerical values) and the qualitative ones (such as safety, lasting fault free 
and strong/weak resiliency). 


Definition 8 (Quantitative problems) Given a CLTS C, 0 < £a < 1,0 <7y<1 
and v € [0, co}: 


— The é-safe active diagnosis problem consists in deciding whether there exists a 
strategy o such that Co is diagnosable and ¢-safe; 

— The (y, v) fault free active diagnosis problem consists in deciding whether there 
exists a strategy o such that C, is diagnosable and (y, v) fault free; 

— The a-resilient active diagnosis problem consists in deciding whether there exists 
a strategy o such that C, is diagnosable and a-resilient. 


Definition 9 (Qualitative problems) Given a CLTS C: 
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— The safe active diagnosis problem consists in deciding if there exists a strategy o 
such that C, is diagnosable and safe; 

— The lasting fault free active diagnosis problem consists in deciding if there exists 
a strategy o such that C, is diagnosable and lasting fault free; 

— The strongly resilient active diagnosis problem consists in deciding if there exists 
a strategy o such that C, is diagnosable and strongly resilient; 

— The weakly resilient active diagnosis problem consists in deciding if there exists 
a strategy o such that C, is diagnosable and weakly resilient. 


Example 8 In order to illustrate the impact of taking into account infinite memory 
strategies, let us examine the CLTS C of Figure 3. The only ambiguous observed 
sequence is a”. A strategy o thus makes it diagnosable iff the probability of this 
observed sequence in Co is 0. However, the only correct run is p = qou(qia)” with 
observation a“. Thus, C is not actively safely diagnosable. 

Denoting as previously by p,, the probability to forbid b after the observed se- 
quence a” given by the strategy o. Then Pe, (qou(aqi)") = 3 [Lien re, Thus, by 
choosing pn = 1 — mani Co is diagnosable, lasting fault free and strongly resilient. 
On the other hand, no finite memory strategy could achieve this goal since otherwise 
by Theorem 1, C would be actively safely diagnosable. 





3.2 Undecidability of the Quantitative Problems 


The quantitative problems related to fault freeness and resiliency turn out to be un- 
decidable. The proofs of these results are obtained by reductions from undecidable 
problems for probabilistic automata, a well-studied model that combines probability, 
control and partial observation, see e.g. [10]. A probabilistic automaton is a finite 
automaton equipped with a probability distribution on the transitions exiting a given 
state and labelled by a given letter. Given a finite word, we obtain a distribution on 
the paths labelled by this word and the acceptance probability of this word is the 
probability of the subset of these paths ending in an accepting state. More formally, 
a probabilistic automaton M = (S, so, F, X, P) is defined by: 


— S, afinite set of states with sọ € S the initial state and F C S the subset of final 
states; 

— X, a finite alphabet; 

— P a matrix S x X x S with rational non-negative coefficients such that for all 
s € Sandalla € X, X yeg P(s,a,s) = 1. 


The acceptance probability of a word w = w1 . . . Wn, valm(w) is defined by: 


n-1 
valu (w) = 5 II P(s;, Wi+1, 8:41). 


S1,...,S8n|S8n EF t=0 


Let 0 < 6 < 1 be an arbitrary threshold. Given M a probabilistic automaton, the 
problems of deciding the existence of a word w such that (1) valyj(w) > 8 or (2) 
valu(w) > 0 are undecidable [6]. In the following reductions, we choose 0 = L, 
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Before developping it, we give a sketch of proof of the next proposition. Given a 
probabilistic automaton M with alphabet X, one builds a CLTS C composed of two 
independent parts each one initially entered with probability 4 by an unobservable 
transition. The unobservable event leading to the first part is the fault f which can 
only be detected almost surely if the observable event { ¢ X occurs with probability 
1. The second part is constituted of a CLTS version of M augmented by exiting 
transitions. One exits M with probability 4 at every step toward a faulty sub-part 
except if the ț event is triggered. In this case, if the system was in a final state of 
M it goes back to the initial state of the automaton instead of performing a fault. If 
there exists a word w with an acceptance probability at least 4, the strategy which 
consists in forcing the observed sequence wi as long as the run stays in M ensures an 
average observable length (without discount) of the maximal correct signalling prefix 
greater or equal to 1. In the opposite case, we show that no strategy can achieve this 
threshold. 


Proposition 1 (7, v)-fault free active diagnosability is undecidable. 


Proof We proceed here by reduction from the problem of the existence of a word w 
such that valm (w) > 4. We consider the probabilistic automaton M = (S, 50, F, X, P) 
for which w.l.o.g. we assume that: (1) X N {u,f,#,4} = Ø and (2) the probabilities 
are fractions 4 with fixed denominator d. One builds the CLTS C = (Q, qo, X”, T} 
described in Figure 5 and defined by: 


-Q=SU {q0 4}, 925925 fi, fo}: 
- D! = Yu {f,u, f, b}, Xu = {f,u} and Xe = X U {#}; 
— the transition function T is defined as follows. 
1. T(go,f, fi) = T(qo,u, 80) = T(ac,t,92) = T(a2, tae) = T(@2,f, f2) = 
T(a@,f, f2) = T (fa, 4, fo) =T( ft fe) = 1; 
2. for every a € X, T(f1,a, f1) = 1; 
3. for every s,s’ € S and every a € X, T(s,a,s’) = d- P(s,a,s’) and 
T(s, a, dé) = d; 
4. for every s € F, T(s,#,q}) = 1 and for every s € S \ F, T(s,t,q2) = 1; 
5. for every other triplet, T is equal to 0. 


As detailed above, the probabilities in M are all multiplied by their common d, to 
obtain integer weights, and we write d - M in the figure to represent this scaling. 
Let us show that A is (1, 1)-fault free iff there exists a word w accepted in M with 
probability at least L, 

Let o be an arbitrary strategy, Co is diagnosable iff & occurs almost surely in a run. 
Indeed an observed sequence w € X* is ambiguous. On the other hand every run p 
leaving SU { fı } almost surely reaches f2 where h occurs and, whatever p, a fault has 
occurred. 

e Assume that there exists w = w1 . .. wg € X* such that valm (w) > 
the deterministic strategy o by: 


= o(w) = {f, u, #, th; 
— for al0 < i < k, o(wı ... wi) = {f, u, wi+1, 9}; 
- o(w’) = X' for any other word w’. 


E, We define 
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Fig. 5 From a probabilistic automaton to a CLTS. 


Observe that after at most k + 1 observable events, any run leaves SU { fı } and thus 
4 occurs almost surely implying that C, is diagnosable. 


By definition of C and øg, a correct signalling run p such that x(p) = w1 ... wi fori < 
k has probability 4 of staying correct at the next step depending on if the current state 
is q2 or belongs to S. Similarly, a correct signalling run p such that 7(p) = w1 ... Wp 
has a probability val m (w) of being at the next step in q} and 1 — valų (w) in q2. 
Moreover, in state q3, a correct signalling run has a probability 4 of staying correct 
and in q? at the next step. 


Therefore for all n € N, we have n < k implies P(C,,) = (4)” and n > k implies 


P(Cn) = ($)"~!val 4 (w) > (5)”. Finally: 51%, P(Cn) > DY EL = 1. 
e Assume that for all w € X*, valu(w) < 4. Let o be a strategy such that C, is 
diagnosable. Observe that (using a slight and understandable abuse of language): 


Po(Cn)= XO P(wAC)+ XO Polotno S> So Po(wt* AC). 


ween wE”! l<k<nwerynr-k 


Po (Cn) 


Let us show that Po (Cn+1) < 5 


with P, (wi) > 0. 


Po(Cn4i)= >. D Po(weAC)+ XO Po(wh? AC)+ 


wee” xEZU{#} wEeryn-1 


SD SO Po(wi**? aC) 


1<k<nwesnr-k 


with a strict inequality if there exists w € Z"—1 








Let us examine the three terms. 

o A correct run p with observed sequence w has a conditional equiprobability that 
last(p) € S or last(p) = q2. Thus, So cn revu Po(wa) = dues Polw). 
o A correct run p with observed sequence w#" such that k > 1 verifies last(p) = q?. 
Thus, Dicien Pwes- Po (wH t! A C) = E D ielén Pwes- Po (w#" A C) 
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o A correct run p of observed sequence wi has a conditional probability valm (w) 
that last(p) = q} and 1 — valm (w) that last(p) = q2. Thus: 


NI = 


XO P(ufAC)= X valu(w)P(wtAC)<= XO P;(wtAC) 


wEeynr-1 weynr-1 wernt 


with a strict inequality if there exists a word w with Po (wt) > 0. 
By assumption, C, is diagnosable. Thus, according to our characterisation of a strat- 
egy ensuring the diagnosis, there exists a word w such that Po (wi) > 0. As a conse- 
quence, Xp; P(Cn) < Dp (4)” = 1, thus A is not (1, 1) fault free. 














Observation. A straightforward adaptation of the proof establishes that for 0 < y < 
1, Ais (+, oa) fault free iff there exists a word w such that valu(w) > 5. 


Proposition 2 a-resilient active diagnosability is undecidable. 


Proof We proceed here by reduction the problem of the existence of a word w such 
that valu (w) > +. We consider the probabilistic automaton M = (S, so, F, X, P) 
for which we assume w.l.o.g. that: (1) X N {u, f, 4,4} = Ø and (2) the probabilities 
are fractions % with d fixed. One builds the CLTS C = (Q, so, X”, T} represented in 
Figure 6 (with some shortcuts to ease readability) and defined by: 


-= Q = SU {a fi} 
- 3! = DUE, tb}, Dy = {f} et 3e = DU LH}: 
— the transition function T is defined by: 


1. EN Giada fı) oa PGi f1) = l; 

2. forall s,s’ € S,a € X,T(s,a,s') = d- P(s,a,s') and T (s, a, q1) = d; 
3. forall s € F, T(s, 4,59) = 1 and for all s € S \ F, T(s,#,q1) = 1; 

4. for every other triplet, T is equal to 0. 


Here again, the probabilities in M are multiplied by the constant d, which we abbre- 
viate in the figure by d - M. 


es i 
f,1 


Z,dU {(#,1)} 





Fig. 6 From a probabilistic automaton to (another) CLTS. 


As a faulty run is followed by a 4, whatever the strategy o, Co is diagnosable. 


e Assume there exists w = w1 ... wp € X* such that valu(w) > +. We denote 
v = valu(w). We define the deterministic strategy o by: 
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- o((wt)*w) = {8,44}: 

— for al 0 < i < k, o((w)* w ... wi) = {£, b, wi4i}s 

- o(w’) = &” for any other word w’. 
Under strategy o, the observed sequence of a correct run p is some (wH ™ wy... wi 
withO < i < k. 
o If r(p) = (wH)™w...w; with 0 < i then with conditional equiprobability, 
last(p) € S or last(p) = qı. Thus with probability à the run will be correct af- 
ter the next observation. 
o If r(p) = (wH)™ then with conditional probability v, last(p) = so and with prob- 
ability 1 — v, last(p) = qi. Thus with probability v, the run will be correct after the 
next observation. 
Consider an arbitrary n and write its Euclidian division by k + 1 as n = m(k+1)+7 
with i < k. One has 2-(-YP,(C,) = (2)" 

—(n—1 wed 

on > (Qo 





Hence implying lim» FT = 0. SoC, is 4-resilient. 


e Assume now that for all word w € Z*, valu(w) < 3. Let ø be an arbitrary 
strategy. The observed sequence of a correct run p is some u1f...fu,, such that for 
all i, u; € X*. 

o Si um Æ 1 With 0 < i then with conditional equiprobability, last(p) € S or 
last(p) = qı. Thus with probability 3, the run will be correct after the next observa- 
tion. 

o If um = 1 then with conditional probability val m (um-—1), last(p) = so and with 
probability 1 — valy4(um_1), last(p) = qı. Thus with probability val m(um-1), 
the run will be correct after the next observation. 


=n 


2 
Po(Cn) 





Summarising one has: P,(C,,) < 2-(*-1) implying limsup, > 


NI 














So C, is not ł resilient. 


3.3 Decidability of the Qualitative Problems 


In contrast to the quantitative notions, and to the notable exception of the safe ac- 
tive diagnosis, all the qualitative problems of diagnosability under degradation con- 
straints we introduced are decidable and EXPTIME-complete. Moreover, to remedy 
the undecidability of the safe active diagnosis problem [2], in a second step, we also 
establish its EXPTIME-completeness when restricted to finite-memory strategies. 

We start with the three qualitative problems that turn out to be decidable, even 
with no restriction on the memory of control strategies. The techniques used in the 
proofs are similar in spirit to the ones used for solving decision problems in Partially 
Observable Markov Decision Processes [5]. The proof idea is common to all cases: 
we establish a necessary and sufficient condition for the existence of a control strategy 
that ensures the given notion of diagnosability under a degradation constraint. To do 
so, we revisit the construction given in [2] (for active diagnosis), which builds an 
enriched model including finite information of the history. On this enriched model, 
the necessary and sufficient condition consists of graph-based properties. 

Let us start by recalling the construction from [2]. To decide the diagnosability of 
a CLTS, its states are enriched with two subsets of states: U and V that correspond, 
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respectively, to the subset of correct, or faulty, states, that are reachable by a signalling 
run corresponding to the current observed sequence. A pair (U, V) is called a belief. 
Formally, from a CLTS C = (Q, qo, X, T), we define its belief version on the same 
event alphabet C? = (QP që, X, TP) by: 
- Q? =Q x 22 x 2% and që = (go; {40}, 0); 
— for every (q, U, V) € Q x 28 x 29, for every a € X, and every q € Q 
- ifa ¢ Xo, T? ((q, U, vV), a, (g', U, V)) = T(q, a, q'); 
— ifa € X7, letting 
1. U' = {q € Qe | Sac € U, 3p € SR, qe > q, A T(P) = a} 




















2, V' = {qf € Qr | dae EUUV, 2 € SRi, qe > qf AT(p) = a}. 


then T?((q,U,V),a, (q’,U',V’)) = T(q,a,q'). 
— for every other triplet ((q, U, V), a, (q’, U”, V’)), T is equal to 0. 


The size of the belief CLTS CË is exponential in the size of C. For the properties we 
are interested in, they have the same behaviour. We introduce A, a discrete version of 
T®, extended to observed sequences. For w € X*, (q’,U’, V") € A((q, U, V), w) as 


soon as there exists a run p such that 7(p) = w and (q, U, V) & (q',U', V^). 


We will now construct Win the set of all beliefs (U, V) such that, starting from 
any (q,U,V) with q € U UV, CP is actively diagnosable. This set is computed as 
a greatest fixpoint. We let Wing = 2%° x 2°F and for n € N, Winy+1 is the set of 
the beliefs (U, V) of Win, such that for all state q € U U V, there exists a sequence 
of sets of allowed events (2% )i<i<k and an observed sequence w = 01... 0, With 
oi € X? verifying: 

— there exists a run p starting in (q, U, V) with 7(p) = w and reaching (q*, U*, V*) 
with q* € Qe (i.e. the current state is correct) or U* = 0 (the fault is claimed); 

— Consider a state q; reached from q’ € U U V by a run with observed sequence 
01...0; WihO < i < k, ie. (qi, Ui, Vi) € A((q',U,V),01...0;) for a belief 
(Ui, Vi). then: 

1. the control induced by Z?,, does not create any deadlock: G7 (qi) Æ 0; 

2. Every new belief obtained by an observable step o € %?,, starting in q; 

belongs to Winn: Vo € 27,1, Y(do, Uo, Vo) € A((qi, Ui, Vi), 0), (Uo, Vo) € 
Winn. 


The computation of Win is in polynomial time in the size of CP, given that at every 
non-terminal iteration at least one belief is removed. The correctness of Win is estab- 
lished in [2], and o* a (deterministic finite-memory) strategy ensuring diagnosability 
consists in, given a belief (U, V) € Win choosing the greatest set X° such that every 
possible belief reached on the next step still belongs to Win. 

To decide weakly (resp. strongly) resilient active diagnosability, and lasting fault 
free active diagnosability, we build on the belief CLTS contruction. The simplest case 
is the weak notion: 


Theorem 2 Weakly resilient active diagnosability is EXPTIME-complete. 


Proof We first establish the membership in EXPTIME. Given a CLTS C, its belief 
CLTS C®, and the deterministic finite-memory o*, we derive a pLTS A. It is obtained 
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from CE by restricting to the states of with belief in Win and controlled by the strategy 
o*. We claim that C is actively diagnosable with guarantee of weak resiliency iff there 
exists in A a reachable cycle such that the first component of every state along the 
cycle is a correct state of C. 

e Suppose first that such a cycle exists in A. We let a > 0 be the probability of this 
cycle, nı its length, no the observed length of the shortest run reaching a state of the 


cycle and y the probability of this run. For all n > no, P4(Cn) > palm, As a 
consequence, À is a’-resilient for all a’ < a. A is thus weakly resilient. Therefore, 
Co», which has the same probabilistic behaviour as A is weakly resilient too. 

e Conversely, suppose that there is no such cycle in A. Let g’ be a (live) strategy such 
that C, is diagnosable. This strategy can be mimicked in CP, ignoring the belief 
information. The reachable states of CË, are associated with beliefs of Win (due to 
the characterisation recalled above). As o* is the most permissive strategy ensuring 
to stay in Win, there does not exist any such cycle in CË, either. Consequently, there 
exists nf € N such that every run p in C 3 with |p| > np ends in a state which first 
component is faulty. Thus Pe, (Cn, ) = Pes (Cn; ) = 0, which means that Co» is not 
weakly resilient. ° 


The complexity lower-bound is obtained by reduction from the active diagnos- 
ability, which is known to be EXPTIME-hard [2]. For C = (Q, qo, X, T} a CLTS, we 
define the CLTS C’ = (QU {q, as}, 96, X U {#4}, T’) with f a fresh observable event, 
and such that T” (q9, #, qo) = T’(q0,#,9s) = T’ (qs, H, qs) = 1, for all g,q’ E€ Q,a € 
X, T'(q,a,q) = T(q,a,q’) and for every other triplet T’(q,a,q’) = 0. Clearly 
enough, C’ is diagnosable iff C is diagnosable. Moreover, C’ is safe by construction, 
and thanks to Theorem 1(a), it is strongly resilient, and thus weakly resilient. 














The proof of the next theorem also relies on the set of beliefs Win. We build a 
subset of Win, called WinK. A belief (U, V) of Win belongs to Wink if there exists a 
strategy o such that from every distribution with support U U V, o guarantees to stay 
in Win, and to give a positive probability to the set of infinite correct runs. The CLTS 
is actively diagnosable with guarantee of strong resiliency iff from the initial belief 
one can reach a belief of WinK while staying in Win. The winning strategy consists 
in combining cleverly the strategy used to make the system diagnosable and the one 
allowing to stay in WinK. 


Theorem 3 Strongly resilient active diagnosability is EXPTIME-complete. 


Proof Let C be a CLTS. As in the construction preliminary to Theorem 2, we build 
CB, Win and o*. We then define WinKy € 2° x Win by a greatest fix point compu- 
tation. For (U’, (U,V) € Winky , (U,V) is a belief for which there exists a strategy 
allowing to a set of runs starting in U’ to stay in the states of C? associated with a 
belief of Win while staying correct. WinKy is obtained as the limit of a decreasing 
sequence (WinK,,),cn defined inductively by: WinKg = {(U’,(U,V)) | (U,V) € 
Win A @ 4 U’ C U} and for n € N, WinK,+1 is the set of elements (U’, (U, V)) of 
Wink,, such that there exist a set of allowed events X° verifying: 


— X° does not create a deadlock: Yq € U UV, G*" (q) 4 0; 
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— under the control X° no run starting in a state of U” will make a fault before the 
next observation: Vgc € U’, Yp € SR1, qe & qaT (pP) E 5° = q € Qe; 
— every triplet reached by an observable step o € Z° belongs to WinK,,: 
(U’, (U,V)) € WinK, with: 
1. U' = {qe € Qe | Ade € Uj, 3p € SR1, qe > de A T(P) = a}; 
2. (U,V) is obtained by the update of the belief (U, V) following the observa- 
tion o. 








From WinKy, we define the set WinK € Win by keeping only the second component 
of WinKy: WinK = {(U,V) € Win | 3U’,(U’,(U,V)) € Winky}. Let us state 
some of the properties of this construction. 





- By induction, if (U’,(U,V)) € WinK, then for every (live) strategy, there exists 
a faulty run starting in U’ of observable length n; 

- If Ø # U” CU' then (U’, (U,V)) € WinKy implies (U”, (U, V)) € Winky. 
Thus, if (U,V) ¢ WinK, for all q € U, ({q}, (U, V)) € Winky. 


We also define PreWin the set of states of C? of the form Q x Win from which a 
state (q, U,V) with (U, V) € Wink is reachable. Let us show that C is diagnosable 
and strongly resilient iff the initial state of C? belongs to PreWin. 


e Suppose that the initial state belongs to PreWin. Let (U’, (U, V)) be an element of 
Winky. We define oa(u (u,v)) the finite-memory strategy with memory states of the 
form (U’, (U, V)) and which, starting from (U’, (U, V)), ensures to stay in Winky. 
This strategy immediately derives from the fixpoint definition of Winky. 

For (U,V) € WinK, we also define o{u,v) = 9(u’,(u,v)) for an arbitrary U” such 
that (U’, (U, V)) € Winky. 

Finally, we let oo be the following strategy working in three successive phases 
which may not all be triggered. 


1. First op mimicks o* until a belief (U, V) € WinK is reached; 
2. Then, at every observed sequence w, do chooses to apply o(y,v) with probability 


Pw = ah and to switch to the third phase with probability 1 — pw; 
3. Finally, oo behaves forever as o*. 


We observe that C,, is diagnosable. Indeed, on the one hand, the events allowed by 
do are included in those allowed by the maximally permissive strategy o*, and on the 
other hand almost-surely, o* is applied from some moment on. Therefore every fault 
will almost surely be detected. 

Moreover, let us prove that it is strongly resilient. Indeed, by definition of PreWin, 
there exists a run p starting in the initial state and reaching a state (q, U, V) such 
that (U, V) belongs to WinK. Let U’ C U the one chosen arbitrarily when defining 
o(u,v). Without loss of generality, we suppose that p reaches a state of U”. As a fault 
can only be created after p if oo switches to its third phase, for n > |p|, we have 








: a Ti lel 
Poo (BE Cn | P 3 À) > Poolo) JI => = Poolo) 
i=|pl 
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a” 


Thus, for every 0 < a < 1, similarly to na”, & (Cay converges to 0. 
oo (Cn 





e Conversely, suppose that the initial state does not belong to PreWin. Let o be a 
strategy ensuring diagnosability. For every state (q, U, V) with q € U reachable by a 
run po with ø, (U, V) € WinK and due to our one of our observations ({q}, (U, V)) € 
Winky. Let K be the number of iterations in the fixpoint computation of Wink. 
Then, for every sequence of K random choices under ø, there exists a faulty run 
p € F, compatible with these choices, starting in (q, U, V) and of observable length 
smaller than K. Adding up the probabilities of runs corresponding to every sequence 
of choices of o we obtain 


Po(p € Fhpolo+K | Po < p) > MIP, (p0) 





where \ = mingeg az Gq: Thus, for every n € N, Po (Cn x) < Po(Cn)(1—-A*!@!). 
Letting a = (1 — AKIRI x, we obtain lim,_, PAGS) > 0, so that C,, is not 
strongly resilient. 


To conclude the proof, we observe that the EXPTIME-hardness derives from the 
same reduction as in the proof of Theorem 2. 














It turns out that this same combination of strategies can be used to ensure last- 
ing fault freeness and diagnosability. In fact, the following theorem establishes that 
the characterisation of the strongly resilient active diagnosability also applies to the 
lasting fault free active diagnosability. 


Theorem 4 Lasting fault free active diagnosability is equivalent to strongly resilient 
active diagnosability. 


Proof We will show here that the characterisation given in the proof of Theorem 3 
for a CLTS to be actively diagnosable with guarantee of strong resiliency also char- 
acterises the fact that the CLTS is actively diagnosable with guarantee of lasting fault 
freeness. This will show the equivalence of the two notions in the active case. 
We reuse the definitions from the proof of Theorem 3. Let us show that C is actively 
diagnosable with guarantee of lasting fault freeness iff the initial state of C? belongs 
to PreWin. 
e Suppose that the initial state belongs to PreWin. Then, as discussed in the proof of 
Theorem 3, Coo is diagnosable and there exists a finite run p such that P(p € C, | 
p 3 à) > P(p) £h. Thus: 

DP()> D PE Cu |? 49 > Poll D> — = 00: 

n=1 n=|p| n=|p| 
e Conversely, if the initial state does not belong to PreWin. Let o be a strategy ensur- 
ing diagnosability. For every n € N, P(Cn+ g) < P(Ch)(1 — A*!@!). Thus: 
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Given the equivalence of strong resiliency and lasting fault freeness, from Theo- 
rem 3 we derive: 


Corollary 1 Lasting fault free active diagnosability is EXPTIME-complete. 


We now turn our attention to safe active diagnosability. The problem is known 
to be undecidable in general, and in NEXPTIME when restricting to finite-memory 
strategies [2]. Note that decidability is not immediate even if the strategies are as- 
sumed to be finite-memory, since no a priori bound on the memory is known. We 
refine that complexity result by proving that safe active diagnosis can be solved in 
EXPTIME when restricting to finite-memory strategies. 

To do so, we prove a more general result in the context of a well-known model, 
quite popular in artificial intelligence and more recently in formal methods, that 
combines partial observation, probabilities and control, namely Partially Observ- 
able Markov Decision Processes (POMDP). We establish that the existence of finite- 
memory schedulers that ensure a Biichi objective with probability 1 and a safety 
objective with positive probability in a POMDP is decidable in EXPTIME. We then 
reduce the safe active diagnosis of a CLTS C restricted to finite-memory strategies to 
the existence of a finite-memory scheduler in a POMDP Mc ensuring at the same time 
a Biichi objective with probability 1 and a safety objective with positive probability. 


Definition 10 (POMDP) A partially observable Markov decision process (POMDP) 
is a tuple M = (Q, qo, Obs, Act, T) where 


— Qisa finite set of states with qo the initial state; 
— Obs : Q — © assigns an observation O € © to each state. 
— Act is a finite set of actions; 
- T : Q x Act — Dist(Q) is a partial transition function. Letting Ena(q) = {a € 
Act | T(q, a) is defined} the set of enabled actions in state q, we assume that: 
— for all q € Q, Ena(q) 4 0, and 
— whenever Obs(q) = Obs(q’), then Ena(g) = Ena(q’) and slightly abusing 
our notation, we will denote by Ena(O) the set of events enabled in every 
state with observation O. 


A decision rule is a distribution from Dist(Act) that resolves non-determinism by 
randomization. A scheduler fora POMDP maps histories of observations to decision 
rules. Formally, a scheduler is a function r : OF — Dist(Act) such that for every 
O1 --- O;, Supp(7(O1--- O;)) C Ena(O;). Given a scheduler 7, a POMDP M yields 
a stochastic process. This stochastic process can be represented by an infinite state 
pLTS, denoted M(r) in which states are histories of observations. One denotes by 
P2 (Ev) the probability that event Ev is realized in this process. 

In the context of POMDP, a belief is a non-empty set of states that represents the 
current state estimate, i.e. the set of states the system may be in, given the actions and 
observations so far. The initial belief is {qo}, and given a current belief B, a decision 
rule ô and an observation O, the belief obtained after ô has been applied and O has 
been observed is defined by: 


A(B, (6,0)) = U Supp(T(q, a)) N Obs 1(O) . 


qEB, aeSupp(ô) 
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Of course, beliefs can similarly be defined for CLTS. Again, the initial belief is 
{qo}, and given a current belief B and an observed event b, the belief obtained after 
b has been observed is defined by: 





A(B,b) = {qE Q| Ad € B,p €SRi,q' & qAn(p) =b}. 


Intuitively, A(B, b) is the set of states a partially observable systems may be in, given 
that the previous belief was B and observation O occurred. It does not depend on the 
strategy as every controllable event is observable. The set of beliefs is denoted Ble 
and we drop the subscript when there is no risk of confusion. Beliefs are of impor- 
tance since they formalize the discrete information an observer has on the current 
state of the system. 

Aiming at providing a POMDP Mc for the safe active diagnosis problems of a 
CLTS C, we face several difficulties. First, in a CLTS the observations are related 
to events while in a POMDP they are related to states. Fortunately, the relevant in- 
formation pertaining to the observations, namely the information about ambiguity of 
observed sequences, is available in the belief. Thus (with one exception) the states 
are pairs of a state q of the CLTS and a belief B. A second adaptation concerns the 
control mechanism. In C, the control is performed by choosing (possibly randomly) 
a subset of allowed controllable events. Thus actions of Mc are subsets of events 
that include the uncontrollable events. Given some control decision X°, to define the 
transition probability of Mc from (q, B) to (q’, B’), one must consider all paths in C 
labelled by events of X° from q to q’ such that the last event is the only observable 
one. The probability of any such path is obtained by the product of the individual step 
probabilities. The latter are then defined by the normalization of weights w.r.t. X°. 
Finally, there cannot be infinite paths of unobservable events due to the convergence 
of C. However some paths can reach, via unobservable events, a state from which no 
event of X° is enabled. In other words, the control X° applied in (q, B) may have 
a positive probability to reach a deadlock (i.e. the chosen decision rule leads to a 
strategy for the CLTS which is not live). In order to capture this behaviour and to 
obtain a non defective probability distribution, we add an additional state lost, that 
corresponds to such deadlocks. The next definition formalizes our approach. 


Definition 11 The POMDP Me = (Q™<, e, Obs, Act, 7™c) derived from a CLTS 
C = (Q, qo, X, T) is defined by: 


- QMe = Q x Ble w {lost} with q)° = (qo, {00}; 

— the set of observations is © = Ble U {lost}, with Obs(lost) = lost and for 
(q, B) € QM, Obs((q, B)) = B; 

- Act = {5°C 5| 2° D E \ Ie}; 

— for every (qı, B) € QM° and X° € Act, T™c ((q1, B), X°) = u € Dist(Q™) 





where: 
- p((q', B’)) = 
n 
5 5 (II TE (qi, ai, 441)) TE (qn41,0, 9); 
oe Pe 7 dE 


a-anEZ NZ 
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n 
- p(lost) = >: Il TE (qi, Qi, qi+1); 
aa ang as 
a1 an EX 'ADu 
CE (dn+1)=0 
- for every X° € Act, TMc (lost, X°) = liost- 


Given C, the construction of Me, which is of size in 20 (QIE) can be done 
in exponential time. Also, the probability distributions over next states (u in Defini- 
tion 11) are presented as sums over paths of C, but they can be computed in polyno- 
mial time by matrix operations (as for DTMC). 

A CLTS C and its associated POMDP Mc are closely related. In particular, strate- 
gies in C and schedulers in Mc are in a one-to-one correspondence. On the one hand, 
let us explain how to naturally derive a strategy o for C from a scheduler 7 in Me. 
For an observed sequence a1-::a, € 2%, there is a unique sequence of beliefs 
Bo,- -- Bn such that Bo = {qo} and for all i < n, Bi+ı = A(Bji,a;41). We then 
set o (a1 `- - an) = T(Bo- -+ Bn). Notice that the strategy o obtained that way is not 
necessarily live: for example, if after B1, ... Bn the choice of ø leads with positive 
probability to lost, then ø is not live. However, as soon as 7 ensures to avoid state 
lost, then the corresponding strategy ø is live. 

On the other hand, to a live strategy o for C, we can associate a scheduler 7 in 
Mc that always avoids lost. For a sequence of observations that does not contain 
lost, thus of the form Bo: -+ Bn, with B; C Q for all i, we pick a1 -+ -an € X3 
an observed sequence such that for all à < n, Biz, = A(B;,a;+1). We then set 
T(Bo::: Bn) = o(a- -an). Note that the observed sequence is not uniquely de- 
fined from Bo - - - Bn. However, if a, --: an and aj --- al, both lead to the belief Bn, 
the set of possible states of the CLTS after both observed sequences is the same. 
Therefore, the same subsets X° after both sequences leave the system live, and the 
same actions X° yield a probability distribution p such that (lost) = 0. 

Moreover, if (o, 7) is a pair of live strategy and corresponding scheduler (that al- 
ways avoids lost), the probability measures Pe, and PMe are essentially equivalent. 
More precisely, the product in Mc with the belief does not change the probability 
measure defined by C,. 

We now show how to decide for POMDP the existence of a finite-memory sched- 
uler that ensures a Biichi objective with probability one and a safety objective with 
positive probability. We use LTL notations to denote sets of paths in a POMDP, such 
as ©, O and O0 for eventually, always and infinitely often respectively. 




















Theorem 5 The problem whether, given a POMDP M with subsets of states F and I, 
there exists a finite-memory scheduler T such that PM (OF) = 1 and PM(QI) > 0 
is EXPTIME-complete. 


























Theorem 5 derives from Propositions 3 and 4 below, that state, respectively, the 
upper bound in the general case, and the lower bound in a particular case, namely for 
the safe active diagnosability under finite-memory strategies. 


Proposition 3 Given a POMDP M with subsets of states F and I, one can decide in 
EXPTIME whether there exists a finite-memory scheduler + such that PM (OOF) = 1 
and PM (OI) > 0. 
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Proof In this proof, the POMDP M = (Q, qo, Obs, Act, T) is fixed, and we use no- 
tation P° (Ev) to denote the probability of the event Ev under scheduler 7 assuming 
that instead of go, the initial state in M is given by the distribution dg € Dist(Q). 

Let us first explain how to compute the following set of pairs of beliefs: 





Win = {(B', B) | B' C I, B' C B s.t. Jr s.t. 
V6o with Supp(ôo) = B, P? (OOF) = 1, and 
V6! with Supp(5,) = B', P¥ (DI) = 1} . 
































Intuitively, Win_: denotes pairs of beliefs such that there exists a scheduler that en- 
sures a Büchi objective almost-surely from the larger belief, and a safety objective 
almost-surely from the smaller one. Note that, in the definition of Win=1, we do not 
require the scheduler 7 to be finite-memory. Given that we consider pairs of beliefs, 
we introduce the following notation: A((B',B),0:) = (4(B', 01), A(B,0:)), 
and similarly for sequences of actions and observations. Also, for X C Q a subset of 
states, we denote by Blcx = {B € Bl | B C X} the set of beliefs contained in X. 


Lemma 2 Let Win. be the greatest fixed point starting from {(q, B', B) € Q x 
Bl x BL| q€ B,B’ CB, B’ CT} of the following operator: 

















W => {(q, By, Bi) | An > 1, Jqo--- dn € Q, Jar, :: an 101 -+ On, 

(By, B2) = A((Bi, Bi), (a1, O1) +++ (an, On)), Va" € Bo, (q’, By, B2) € W, 

go =q, qn € F, Vi < n, T (qi, i141) (Gi41) > 0,V1 < j < n, Obs(q;) = O;, 

Vi < n, VO!, for (BS, Bs) = A((B', B), (a1, O1) -+ (ai—1, Oi—1)(@;, O;)) 
we have Vq' € Bs, (q, B3, B3) CW NQ x Bic: x BI}. 





We have Win=1 = {(B’, B) | Yq € B, (q, B’, B) € Wing}. 


Proof (of Lemma 2) To establish that Win_; corresponds to the projection on the 
pair of beliefs of Wins, we first assume that for all q € B, (q, B’, B) belongs to 
Wins, and exhibit a scheduler 7 that witnesses (B’, B) € Win=1. Let us define 7 as 
follows. The scheduler 7 has finite memory BL x BL. From memory state (B’, B), T 
dictates to play uniformly all actions a such that for every observation O and every 
q € A(B, a, O), we have (q, A((B’, B), a, O)) € Wing. Note that this set of “safe” 
actions is necessarily non empty because (q, B’, B) € Wina. If a is played, and O 
is observed, the memory state of 7 is updated to A((B’, B), œa, O), which is still in 
Win, by assumption on a. The scheduler 7 then continues similarly with memory 
state A((B’, B), a, O). 

So defined, let us show that 7 witnesses (B’,B) € Winey. First, let do be a 
distribution with support B. The scheduler 7 ensures to stay (surely) in Win... More- 
over, for every q € B, with a positive probability, say P(4,B’,B) > 0, the sequence 
(a1, O1) -++ (an, On) of actions and observations leading to F that derives from the 
fixpoint definition, happens from q. There are finitely many pq, 57,3), all are positive, 
so they are lower bounded by some positive value p. Playing 7 forever thus ensures 
visiting F almost surely, and iterating this reasoning, even visiting F infinitely of- 
ten with probability 1. Now, assuming B’ # () let 64 be a distribution with support 
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B'. Any action picked by 7 ensures that, whatever the observation, the first belief- 
component remains in J. Therefore, surely, from distribution 66 the plays stay in the 
invariant J. 

Let us now assume that the triplet (q, B’, B) is removed during the iterative com- 
putation of the fixed point Was. We prove, by induction on k, that if (q, B’, B) is 
removed at iteration k, then, (B’,B) ¢ Win=1. If k = 0, the pair is removed at 
initialization, hence B’ Z I or B’ Z B, and obviously (B’,B) ¢ Win zy. Oth- 
erwise it happens at the k-th iteration, for some k > 1. Assume, towards a con- 
tradiction, that there exists a scheduler 7, witnessing that (B’,B) € Wine. In 
particular, there exists a sequence of pairs of actions and observations allowed by 
the scheduler (a1,O01)---(a,, On) so that there exists go...dn € Q with go = 
q, qn € F, Vi < n, T (qi, Qi41)(Gi41) > 0,Y1 < j < n and Obs(q;) = O;. 
Because the triple (q, B’, B) was removed at iteration k, it must be that, either (1) 
for (BS, B2) = A((B’, B),(a1,01)-++(an,On)), there exists gz € Bo such that 
(q2, B', B) € Wy-1, (2) no path corresponding to a sequence (a1, O1) -++ (an, On) 
satisfying (1) and starting in q ends in F or (3) there exists an index à and an obser- 
vation O/, such that for (BS, B3) = A((B’, B), (a1, 01) +++ (ai—1, Oi—1)(@i, O4)) 
there exists q € Bs, (q, B4, Ba) € Wx-1N Q x Blcr x BL. In the first case, it 
means that either there is a positive probability, under 7 to reach a pair of beliefs out 
of W;,-1, and thus out of Win, by induction hypothesis. As the sequence of action 
and observations was chosen so that one can reach F from q, the second case implies 
that the first case holds with our selected sequence of actions and observations. For 
the third case, let (BS, B3) = A((B’, B), (a1, O1) -++ (ai—1, O;-1) (ai, Of)). Either 
there exists q’ € B3 such that (q, BS, Bs)  Wp—1, then it is treated similarly to the 
first case. Else B} ¢ Blcr. Observe that, in this case, the second requirement on 7 is 


not satisfied since pa ( I) <1. 


























Thanks to Lemma 2, Win_, can be computed in EXPTIME. Let us now define 
Lose as the set of beliefs that are clearly losing: 


Lose = {B € Bl | 747 Yôo with Supp(do) = B, P (OOF) = 1} . 

















As established e.g. in [3] in the more general framework of 2-player stochastic games 
with signals, Lose can also be computed in EXPTIME. 

Informally, we now consider the set of beliefs from which one can reach, while 
staying in J, and not risking to fall in Lose, some belief B such that there exists 
B' + Ú with (B’, B) € Win ). Formally, let Win be the following set of beliefs: 


Win = {Bo € Bl | 3(B', B) € Wine, st. B’ Æ 0 and 
day "An; 20: Te On, A(Bo, (a, O1) ESN (an, On)) =B 
Vi < n, YO}, A(Bo, (a1, O1) re (@i-1, O;-1) (a4, O;)) ¢ Lose}. 











The set Win characterizes winning beliefs, that is, beliefs from which there exists a 
finite-memory scheduler ensuring at the same time, the Büchi objective OF almost- 
surely, and the safety objective LIT with positive probability. Formally: 


























Lemma 3 Bo € Win if and only if for every ĉo with Supp(ôo) = Bo, there exists a 
finite-memory scheduler 7 such that P% (OF) = 1 and P® (OT) > 0. 
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Proof (of Lemma 3) Assume first that Bp € Win. We design a finite memory sched- 
uler 7 that is winning from any initial distribution 69 with support Bo. In a first mode, 
7 aims at reaching a pair of beliefs (B’,B) € Win_, from Bo. More precisely, 7 
plays the path that leads from Bo to some B € Bl such that there exists B’ 4 Q 
with (B’, B) € Winey. If this succeeds, 7 then switches to another mode, where it 
behaves as the winning scheduler that starts from (B’, B) in Lemma 2. If it fails, the 
play ends in a belief Bı ¢ Lose (by definition of Win), and from there 7 plays to en- 
sure visiting F infinitely often with probability 1. All in all, 7 ensures almost surely 
visiting F infinitely often, and with positive probability (the probability of the prefix 
leading to B, times the probability that the play is in B’ at that time point) to stay in 
I. Note that the size of the memory 7 uses is in O(|Bl|?). 

Let now ôo be an initial distribution with support Bo, and assume that there exists 
a finite-memory scheduler 7 such that P% (OF) = 1 and P% (O7) > 0. We consider 
M- the Markov chain generated by 7, with finite state space Q x Mem, where Mem 
is a finite set of memory states. Without loss of generality, we iteratively tag each 
state of M, with its associated belief. Since 7 is winning, there must exist a BSCC 
C in M,, reachable from some (qo, mo, Bo) via an I-path (a path where all belief 
tags are included in J), and such that all states (q, m, B) € C satisfy q € I, and there 
exists a state (qr, my, By) € C such that qf € F. Pick any state (g,m,B) € C. 
From (q, m, B), under scheduler 7, all plays stay in J. Moreover, for any q’ € B, 
from (q’,m, B), under scheduler 7, almost all plays visit F infinitely often. As a 
consequence, by the definition of Win=1, ({q}, B) € Win =1. Then, we conclude that 
Bo € Win, exploiting the I-path from (go, mo, Bo) to C (and thus to any of its states), 
and the fact that 7 ensures OF almost-surely, and thus always avoids Lose. 


















































Win characterizes the winning beliefs, and can be computed in EXPTIME. We thus 
showed the computability in EXPTIME of the set of supports B from which there 
exists a finite-memory scheduler 7 such that P? (OOF) = 1 and PË (OT) > 0. 






































Now the safe active diagnosis restricted to finite-memory strategies can be re- 
duced to the existence for POMDP of a finite-memory scheduler that ensures a Büchi 
objective almost surely, and a safety objective with positive probability. As Mc is 
exponential in the size of C and the algorithm on the POMDP is in EXPTIME, we 
obtain a 2EXPTIME complexity upper-bound. Fortunately, in order to avoid a doubly 
exponential blowup and to establish the EXPTIME complexity, we observe that the 
exponential comes in both cases from the computation of beliefs depending only on 
the original CLTS. This implies that the safe active probabilistic diagnosis problem 
is in EXPTIME when restricted to finite-memory strategies. 


Corollary 2 The safe active diagnosis problem restricted to finite-memory strategies 
is decidable in EXPTIME. 


Proof Given a CLTS C, we build Me and decide if {qo} is a support from which 
there exists a scheduler 7 ensuring pint ( QF) = 1 and pins ( I) > Owith J = 
{(q, B) | q € Qc} and F = {(q, B) | B C Qf Va € Qc}. Due to the link between 
Mc and C, this choice of F corresponds to runs that are either correct or surely faulty 
in C and this choice of J corresponds to runs that are correct. Thus there exists a 
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finite-memory scheduler 7 as defined above iff the corresponding strategy o in C 
ensures safe active diagnosis. Moreover, as explained above the corollary, deciding 
the existence of this scheduler can be done in EXPTIME. 














A matching lower-bound is already known from the literature: 


Proposition 4 ([2]) The safe active diagnosis problem restricted to finite-memory 
strategies is EXPTIME-hard. 


Obviously, this lower bound also holds for the more general problem: on POMDP, 
whether there exists a finite-memory strategy ensuring a Biichi objective almost- 
surely and a safety objective with positive probability. 


4 Conclusion 


We have studied the active diagnosis of partially observable probabilistic transition 
systems combined with some degradation control. More precisely we have intro- 
duced two new notions of degradation both in a qualitative and a quantitative ways. 
We have established their links with the notion of safety in the finite, infinite and 
finite controllable cases. Afterwards we have proved that the quantitative versions of 
the corresponding decision problems were undecidable. Contrary to the safe active 
diagnosis, the qualitative versions of these problems are EXPTIME-complete even 
though the associated diagnosers may require infinite memory. 


We now have a set of algorithmic results both in the passive and active framework 
which could justify the development of a tool. At first, this will require to choose 
and study a more appropriate formalism than probabilistic transition systems from 
a modelling point of view. Another direction would consist in studying a different 
notion of faulty runs. Here a run is faulty once a fault has occurred. A fault could 
only represent a degradation of the system which can still be partially available. In 
this alternative framework, the degradation to be evaluated would be the evolution of 
the number of faults in a run w.r.t. its length. 
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